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algebra.  For  this  reason  all  theorems  in  the  paper  will  be  proved 
by  using  only  the  basic  concepts  of  linear  algebra.  In  addition,  for 
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Properties  of  a  Generalized  Inverse 
with  Applications 
to  Linear  Programming  Theory 


ABSTRACT 


In  the  first  five  sections  of  this  paper  various  properties  of  a  Rao 
generalized  inverse  of  a  matrix  are  established.  A  method  of  computing 
such  an  inverse  is  also  given.  In  order  to  illustrate  the  differences  be¬ 
tween  the  Rao  and  other  generalized  inverses,  a  survey  of  results  on 
Penrose- Moore  inverses  is  included. 

The  last  three  sections  are  devoted  to  showing  how  a  generalized 
inverse  can  be  used  in  the  theoretical  development  of  the  simplex  and 
modified  simplex  methods  of  linear  programming.  In  particular,  it  is 
shown  that  the  fundamental  equations  and  iteration  formulas  of  these  meth¬ 
ods  can  be  derived  using  matrix  notation  without  requiring  the  assumption 
that  the  linear  programming  problem  has  no  redundant  constraints. 
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1.  INTRODUCTION 


This  paper  presents  some  properties  and  applications  of  a  Rao  general¬ 
ized  inverse  AM  of  an  arbitrary  matrix  A.  After  both  ¥  and  the  better-known 
Penrose-Moore  inverse  A*  have  been  defined  and  the  mair  results  and  refer¬ 
ences  on  Penrose-Moore  inverses  have  been  summarized  briefly,  A*  is  shown 
to  be  just  one  of,  in  general,  an  infinite  number  of  A».  With  several  important 
properties  of  A"  established,  the  computational  aspect  of  both  the  Rao  and 
Penrose-Moore  inverses  is  discussed,  and  a  numerical  example  shows  how 
much  easier  it  is  to  compute  A^  than  A*. 

Some  uses  of  A«  in  linear  programming  theory  will  then  be  considered. 
Throughout  the  literature  on  linear  programming,  e.g.,  in  the  development  of 
the  simplex  and  modified  simplex  methods,  theorems  are  proved  by  using  the 
inverse  of  a  basis  matrix  for  the  space  spanned  by  the  activity  vectors  P  , 
j  =  1,  .  .  .  ,  n  and  the  stipulations  vector  P0.  In  general,  however,  such  an 
inverse  need  not  exist.  In  the  distribution  and  network  problems,  for  example, 
a  basis  is  not  square  and  hence  has  no  inverse.  In  fact,  for  any  problem  in 
which  redundant  constraints  exist,  a  basis  matrix  will  not  be  square. 

Here  it  is  shown  that  linear  programming  theorems  can  be  proved  just  as 
easily  by  using  a  left  inverse  of  a  basis  matrix  as  by  using  the  ordinary  inverse. 
It  will  be  shown  that  such  a  left  inverse  always  exists  and  reduces  to  the  regular 
inverse  in  the  event  that  the  basis  matrix  is  square.  It  will  also  be  proved  that 
even  though  the  left  inverse  is  not  unique  it  can  still  be  used  to  give  a  unique 
expression  for  any  P,  in  terms  of  the  basis.  Thus  matrix  equations  of  the  form 
B.X,  Pj ,  where  B  is  a  basis,  can  be  solved  without  considering  whether  B  is 
square. 


2.  RAO  AND  PENROSE-MOORE  INVERSES 

In  1962,  C.  R.  Rao,  an  Indian  statistician,  published  a  paper1  in  which  he 
defined  a  generalized  inverse  (g.i.)  of  a  matrix  as  follows. 

Definition  1— A  Rao  g.i.  of  an  m  xn  matrix  A  is  any  n  >  m  matrix  A#  such 
that  for  any  b,  for  which  AX  b  is  consistent,  X  A«b  is  a  solution. 

Rao  defined  such  a  g.i.  in  order  to  be  able  to  use  matrix  notation  and  theory 
in  solving  systems  of  linear  equations  of  the  form  AX  b  in  cases  where  A  is 
rectangular  or  square  and  singular.^  The  question  of  how  to  define  and  use  a 
g.i.  for  such  problems  has  received  attention  from  time  to  time  in  the  literature. 
In  particular,  attempts  have  been  made  to  define  a  g.i.  with  properties  similar 

tRao  was  particularly  interested  in  using  a  g.i.  for  applications  to  least-squares 
theory. 
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to  those  of  an  inverse  of  a  nonsingular  matrix.  The  first  such  attempt  was  made 
by  Moore.2’3  The  essence  of  his  definition  of  a  g.i.  is  as  follows: 

Definition  2— An  n  x«i  matrix  A+  is  a  g.i.  of  an  m  *n  matrix  A  if  AA+  =  pro¬ 
jection  on  the  range  of  A  and  A+A  =  projection  on  the  range  of  A+. 

Moore  established  the  existence  and  uniqueness!  of  A+  for  any  A  and  gave 
an  explicit  form  for  A+  in  terms  of  the  subdeterminants  of  A  and  AT,  the  trans¬ 
pose  of  A.  Various  properties  of  A*  and  relations  between  A,  AT,  and  A+  were 
presented  by  Moore.3  At  the  same  time  a  variant  of  these  results  was  obtained 
and  given  an  algebraic  basis  by  von  Neumann.4 

Tseng5’6  extended  Moore’s  results  to  closed  linear  operators  on  a  Hilbert 
space.  Further  extensions  of  the  work  of  both  von  Neumann  and  Tseng  were 
obtained  by  Ben-Israel  and  Charnes.7’8 

Because  the  unique  notations  employed  by  Moore  were  not  adopted  by  other 
mathematicians,  his  work  remained  virtually  unknown.  As  a  result,  other  vari¬ 
ants  of  a  g.i.  were  discovered  independently  by  Bjerhammar9*10  and  Penrose.11’12 
The  former  constructed  A+  by  identifying  it  with  a  submatrix  of  the  inverse  of 
a  particular  square  nonsingular  matrix.  He  also  showed  that  the  general  solu¬ 
tion  of  the  matrix  equation  AX  =  b ,  when  consistent,  is  X  «  A+b  +  (I  -  A+A)Y,  where 
Y  is  an  arbitrary  nxl  vector.  Thus  when  Y  is  chosen  to  be  the  null  vector,  it  is 
seen  that  A’  also  satisfies  Definition  1.  Hence  A+  is  also  an  At 

Meanwhile  Penrose  defined  a  g.i.  in  a  manner  that  can  be  shown  to  be 
equivalent  to  Definition  2.  His  definition  is  stated  as  follows. 

Definition  3— A+  is  the  solution  of  the  equations 

AXA  =  A 
X  A  X  -  X 
(AX) r  =  AX 
(XA)T=XA. 

Penrose’s  proof  of  the  existence  and  uniqueness  of  A+  is  based  on  the 
vanishing  of  a  finite  polynomial  in  AT A.  An  extension  of  his  results  to  the  case 
of  infinite  matrices  was  obtained  by  Ben-Israel  and  Charnes,  as  is  noted  in  Ben- 
Israel  and  Wersan.13 

Some  other  results  obtained  by  Penrose  are  summarized  below. 

Theorem  1 

(a)  (A')  +  =  A. 

(b)  (AT)*  (A*)T 

(c)  det  (A)  4  0  implies  A+  =  A-1,  the  inverse  of  A. 

(d)  (ATA)f  A'(AT)+  even  though  in  general  fAB)+  i  B+A+. 

(e)  The  ranks  of  A,  ATA,  A+,  A+A  are  all  equal  to  the  trace  of  A+A. 

Theorem  2 

A  necessary  and  sufficient  condition  for  the  solvability  of  AXB  =  C,  is  that 

AA’CB'B  =  ( , 

tUniqueness  is  the  property  of  A+  that  makes  it  much  more  difficult  to  compute 
than  Ah. 
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in  which  case  the  general  solution  of  AXB  =  C  is 

X  -  A4(  Bf  +  (V  -  A'AVBI)4) 

where  Y  is  an  arbitrary  matrix  of  the  same  size  as  X. 

Theorem  2  is  valid  for  any  A+,  B+  that  satisfy  AA+A  -  A  and  BB+B  *-  B , 
respectively.  It  will  be  shown  subsequently  that  the  condition  that  A+  satisfy 
AA+A  =  A  is  equivalent  to  Definition  1,  so  that  AAWA  =  A  for  any  Rao  g.i.  A*.  Thus 
Theorem  2  will  be  seen  to  hold  for  any  Rao  g.i.’s  A#,  B#. 

Moreover  it  is  worth  noting  that  there  is  a  direct  relation  between  the  prop¬ 
erty  AA#A  =  A  and  the  work  of  von  Neumann4  on  regular  rings.  In  this  work,  von 
Neumann  defined  a  ring  R  as  a  regular  ring  if  and  only  if  for  each  element  a  of 
R  there  exists  an  element  x  of  R  such  that  axa  =  a.  An  excellent  summary  of 
von  Neumann’s  work  appears  in  McCoy  (Ref  14,  pp  147-49). 

Penrose12  suggested  applications  of  the  g.i.  in  least-squares  solutions  to 
inconsistent  linear  equations.  This  idea  was  also  used  by  Tseng,5'8  whose  re¬ 
sults  suggest  the  following  definition  of  a  g.i.:  This  definition,  also  suggested 
by  Penrose  (Ref  12,  p  18),  rests  on  the  least-square  character  of  solutions  to 
linear  equations  obtained  by  using  A+. 

Definition  4— Consider  the  system  of  linear  equations  AX  -  b, where  A  is 
mx  n.  Among  all  virtual  solutions  Xb  of  AX  =  b,  defined  by 


I  I  AXb  -  b  |  I  -  inf  |  |  AX -  b  |  I, 
XfE" 


where  E"  is  n -dimensional  Euclidean  space,  there  is  a  unique  extremal  virtual 
solution  Xg  ,  defined  by  1 1  Xg  |  |  s  inf  [  |  |\fc  1 1  :  Xb  satisfies  the  above  relation}. 

The  g.i.  A+  of  A  is  the  matrix  corresponding  to  the  linear  transformation  of  b 
into  Xg  as  b  varies  in  Em  . 

This  least- square  property  of  A4b  was  used  by  Bjerhammar9’10  in  geodetic 
applications.  In  particular  he  used  it  in  adjusting  observations  that  gave  rise 
to  singular  or  ill-conditioned  matrices. 

Penrose  also  gave  two  methods  for  computing  A+ ,  one  of  which  is  based 
on  a  partition  of  A  that  gives  an  expression  for  A+  in  terms  of  the  regular  in¬ 
verses  of  the  partitioned  submatrices.  The  other  method  is  an  iterative  pro¬ 
cedure  involving  the  subdeterminants  of  A^A.  Improved  methods  for  computing 
A+  can  be  found  in  Ben-Israel  and  Wersen.13 

Den  Broeder  and  Charnes15  gave  the  following  explicit  expressions  for  A+ 
as  a  limit  of  a  sequence  of  matrices. 

Theorem  3 

-  n 

For  any  square  matrix  A,  lim  £  AT  f|  +  AAT)-k  exists  and 

H  ■+  fc  ■  1 

oc 

A+  =  I  A1  (I  4  AA  j)~fc, 
fe  =  1 

where  AT  may  not  be  removed  from  the  series  as  a  factor. 
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Theorem  4 


For  any  square  matrix  A, 

\'  -  I  in.  A1  (AAI  .  A  A  . 

A  .0 

Without  loss  of  generality,  only  square  matrices  A  need  be  considered  in 
Theorems  3  and  4.  For  any  m  x  ti  matrix  B  can  be  written  as  a  square  matrix  A 
by  adding  the  right  number  of  zero  rows  or  columns.  Then  the  n  *  m  g.i.  B+  is 
obtained  from  the  corresponding  submatrix  of  A+. 

Results  relating  A+  to  the  principal  idempotents  of  A  and  the  spectral  de¬ 
composition  of  A  have  been  obtained  by  Penrose11’12  and  also  by  Wedderburn.16 
Festenes17’18  also  developed  a  spectral  theory  for  arbitrary  w«ii  matrices.  In 
aoing  so,  he  used  A+  to  obtain  theorems  on  structure  and  some  properties  of 
matrices  relative  to  “elementary  matrices”  and  the  relations  of  “^-orthogonality” 
and  “^-commutativity.” 

Greville19  gave  an  iterative  procedure  for  calculating  A4  using  successive 
partitions  of  A.  Moreover  (see  Ref  20)  he  gave  a  very  outstanding  account  of 
some  of  Moore’s  results,  following  the  original  Moore  approach.  Greville  also 
used  the  least- squares  properties  of  A+  in  regression  analysis. 

A  review  of  some  definitions  and  applications  of  A+  in  explicit  solutions 
of  systems  of  linear  equations  was  given  by  Bjerhammar21  with  numerical  ex¬ 
amples  and  statistical  applications. 

Pyle22  and  Cline23  considered  applications  of  A+  to  systems  of  linear  equa¬ 
tions.  Their  work  in  this  field  originated  in  Charnes’  seminar  course  at  Purdue 
University  in  1955.  Pyle  used  the  projections  AA+  and  A+A  in  a  gradient  method 
for  solving  linear  programming  problems.  These  projections  were  also  used 
by  Rosen24’25  in  his  conjugate  gradient  method  for  solving  linear  and  nonlinear 
problems. 

Bott  and  Duffin26  defined  an  interesting  g.i.  that  is  closely  related  to  the 
von  Neumann  definition  of  regularity  in  a  ring.4  They  used  their  g.i.  in  the 
analysis  of  electrical  networks  by  Maxwell’s  method. 

Ben-Israel  and  Charnes,27  following  on  Bott  and  Duffin,  have  used  the 
Penrose-Moore  g.i.  in  the  analysis  of  electrical  networks  and  obtained  the  ex¬ 
plicit  solution  of  a  network  in  terms  of  its  topological  and  dynamical  character¬ 
istics.  They  also  pointed  out  the  partially  nonoverlapping  character  of  the  Bott- 
Duffin  and  Penrose-Moore  g.i.’s. 

Further  applications  of  A4  have  been  found  by  Kalman28’29  and  Florentin30 
in  control  theory  by  using  its  least-squares  properties  in  the  mean  square  error 
analysis. 

More  recently  A4  has  been  used  by  Charnes,  Cooper,  and  Thompson31  to 
resolve  questions  of  the  scope  and  validity  of  so-called  “linear  programming 
under  uncertainty”  and  to  characterize  optimal  stochastic  decision  rules  for 
n -period  problems.  Charnes  and  Kirby32  use  the  Rao  g.i.  in  a  discussion  of  the 
consistency  of  the  constraints  of  certain  chance-constrained  programming 
problems 

This  completes  the  brief  summary  of  work  done  on  Penrose-Moore  inverses. 
For  a  more  complete  survey  of  the  field  the  reader  is  referred  to  Ben-Israel 
and  Charnes.8 
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3.  SOME  PROPERTIES  OF  AH 

Now  that  a  brief  re'sume'of  the  major  properties  of  A  +  has  been  presented 
tae  derivation  of  many  corresponding  properties  for  the  Rao  g.i.  A"  will  be 
considered.  For  the  sake  of  completeness  the  concepts  of  elementary  linear 
algebra  needed  in  the  proofs  of  the  theorems  are  defined.! 

Definition  5— A  vector  space  V  is  a  set  of  elements  called  “vectors”  sat¬ 
isfying  the  following  axioms. 

1.  To  every  pair  u  and  r  of  vectors  of  the  set  there  corresponds  a  vector 
v  +  u  in  such  a  way  that 

(a)  v  +  u  =  u  +  v,  i.e.,  addition  is  commutative. 

(b)  v  +  (u  +  uj  =  (v  +  u)  +  u,  i.e.,  addition  is  associative. 

(c)  A  unique  vector  0  exists  such  that  r  +  0  =  v  for  every  :  in  V. 

(d)  To  every  vector  i>  a  unique  vector  -v  corresponds  such  that 
v  +  (-  i')  =  0. 

2.  For  every  scalar  a  and  vector  v  in  the  set  a  vector  a  r  corresponds 
such  that 

(a)  0f(j3r)  =  {a  8)  v  for  any  scalar  8,  i.e.,  multiplication  by  scalars  is 
associative. 

(b)  1  x  if  =  v  for  every  v. 

3.  (a)  a(i>  +  u)  =qm  a u,  i.e.,  multiplication  by  scalars  is  distributive 
with  respect  to  addition  of  vectors. 

(b)  (a+  8)v  =  ar  +  /3c,  i.e.,  multiplication  by  vectors  is  distributive 
with  respect  to  scalar  addition. 

Definition  6— A  vector  subspace  L  of  a  vector  space  \  is  a  collection  of 
vectors  in  V  such  that 

(a)  If  i’cL  then  so  is  at'  for  any  real  scalar  a. 

(b)  If  i> ,  u  €  L  ,  then  so  is  (u  +  c )  e  L . 

It  follows  that  if  t  ],  t'o,  .  .  .  ,i'/;  e  L  then  so  is  every  linear  combination 

of  these  vectors,  i.e.,  E|;  ,  f  L  for  any  set  of  scalars  a  ,,  j  =  1,  .  .  .  ,  (?. 

Definition  7— A  collection  of  vectors  r , ,  .  .  .  ,  r,.  cL  is  said  to  “span” 
(generate)  the  subspace  L  if  for  every  vector  mcL  there  exists  a  set  of  scalars 
a  |,  .  .  .  ,  ak  such  that  u  =  E|;  ,  a ,  c( . 

Definition  8— A  set  of  vectors  r,,  .  .  .  , cb  e  L  is  said  to  be  “linearly 
independent”^!;^  ,  of(  r  =  0  implies  o  ,  =  0  for  all  j. 

Definition  9— A  basis  15  for  a  vector  subspace  L  is  a  set  of  linearly  inde¬ 
pendent  vectors  in  L  that  also  spans  L. 

It  is  clear  that  the  number  of  vectors  in  any  basis  of  L  is  unique.  It  also 
follows  that  if  15  =  (i  j,  .  .  .  ,  r,.j  is  a  basis  for  I-  and  if  u  is  any  vector  in  I  , 

then  u  has  a  unique  expression  u  =  a,  c  as  a  linear  combination  of  c.  , 

j  =  1,  .  .  .  ,  k,  i.e.,  the  values  of  a  ,  j  =  1 ,  .  .  .  ,  fc  are  unique. 

Definition  10— Let  A  be  an  m  ■  n  matrix.  The  “range”  of  A,  RfA) ,  is  the  set 
of  all  m  xl  vectors  b  such  that  there  exists  a  solution  \  to  the  system  of  linear 
equations  AX  -  b,  i.e.,  be  Rf A)^  3  \  )  \\  -  b. 

Definition  11— Let  A  be  an  m  *  u  matrix.  The  null  space  of  A,  \  f A ) ,  is  the 
set  of  all  n  x  1  vectors  \  such  that  A\  =  0,  i.e.,  \  e  \  f  A)  x  AX  -  0. 

t All  theorems  will  be  proved  using  techniques  of  elementary  linear  algebra.  Al¬ 
though  shorter  and  more  sophisticated  proofs  arc  possible,  it  is  more  appealing  to  derive 
our  results  from  first  principles. 
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It  is  easy  to  show  that  R(A)  is  a  vector  subspace  of  m-dimensional  Euclid¬ 
ean  space  Em,  and  \(A)  is  a  subspice  of  n -dimensional  Euclidean  space  E», 
where  Ek  is  defined,  for  any  integer  k,  by 

I  V  :  \r  "  (  x, ,  .  .  .  ,rfe) .  a  real  number  I  . 

Definition  12— The  rank  of  a  matrix  A,  written  “rank  (A),"  is  the  number 
of  vectors  in  any  basis  of  R(A). 

Definition  13— The  nullity  of  a  matrix  A,  written  “nullity  (A),”  is  the  num¬ 
ber  of  vectors  in  any  basis  of  \(A). 

By  Definition  10,  b  c  R  f A)  if  and  only  if  b  can  be  expressed  as  a  linear  com¬ 
bination  of  the  columns  of  A.  Hence  it  is  seen  that  a  definition  of  rank  that 
is  equivalent  to  Definition  12  is  that  the  rank  of  A  is  the  maximum  number  of 
linearly  independent  columns  of  A.  This  can  also  be  shown  to  be  equal  to  the 
maximum  number  of  linearly  independent  rows  of  A  (Ref  33,  p  56). 

Theorem  5 

Assume  that  the  system  of  linear  equations  AX  -  b  is  consistent. 

Then  A*  is  a  Rao  g.i.  of  A  if  and  only  if  AA“A  =  A. 

Proof:  Suppose  AA^A  «=  A. 

Then  AX  =  b  implies  AAwb  -  AAWAX  =  AX  -  b. 

Hence  X  -  A«b  is  a  solution  of  AX  -  b  ,  so  A#  is  a  Rao  g.i.  by  Definition  1. 

Conversely  suppose  A*  is  a  Rao  g.i.  of  A. 

Let  b  =  ,  the  j th  cHumn  of  A. 

ThenAX«b-flj  is  consistent,  hence  X  =  Att0j  is  a  solution. 

Therefore  0|  =-  AX  -  AArt  a  ( . 

Since  this  must  hold  for  all  a,  ,  j  =  1,  .  .  .  ,  n,  A  =  AAHA. 

Using  this  theorem,  it  can  be  seen  that  the  following  definition  is  equivalent 
to  Definition  1. 

Definition  14— An  tixin  matrix  A#  is  a  Rao  g.i.  of  an  mxti  matrix  A  if  and 
only  if  AAHA  =■  A. 

From  this  definition  and  Definition  3,  it  can  be  seen  that  A+  is  also  a  Rao 
g.i.  Thus  all  the  properties  of  A"  that  will  be  established  for  A#  will  also  hold 
for  A+. 

The  following  is  required. 

Lemma  l.f  Let  A  be  an  m  v  n  matrix. 

Then  n  =  rank  (A)  +  nullity  (A). 

If  A  is  tixn  then  RTA)  and  N'A)  are  both  vector  subspaces  of  E".  Moreover 
it  can  be  seen  from  Definitions  10  and  11  that  the  null  vector  is  the  only  vector 
common  to  R'A)  and  MfA);  yet,  every  vector  XeE"  must  be  in  one  of  these  two 
subspaces,  as  either  AX  -  b  /0  or  AX  -  0.  Hence,  except  for  the  0  vector,  R'A) 
and  \'A)  partition  E"  when  A  is  tixn.  This  fact  will  be  used  below. 

In  proving  Theorem  6  the  following  will  be  used. 

Definition  15— An  ti  >  n  matrix  II  is  said  to  be  “idempotent”  if  II2  =  H. 

Lemma  2.  Let  the  «  *  n  matrix  II  be  idempotent. 

Then  nullity  fll)  =  rank  (I  -  II) ,  where  I  is  the  iixii  identity  matrix  (i.e. , 
l\  =  X  for  every  Xe  EM). 

|A  proof  of  this  lemma  can  be  found  in  Perlis  (Ref  33,  p  54). 
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Proof:  Let  Ye  Rfl  -  II). 

Then  there  exists  X  such  that  fl  —  1 1 )  \  Y. 

Hence  IIY  II  fl  -ll)X  -  fll  _  II 2)  X  0,  so  Yc  \  fll). 

Conversely,  let  Ye \  fll)  and  Y  /  0. 

Suppose  YeXfl  -II).  Then  fl  —  II) Y  =0  Y-  IIY  =  Y  as  YeV fll)  .  Therefore 
Y  =  0  and  a  contradiction  exists. 

But,  by  the  remark  following  Lemma  1,  Rfl  -II)  and  \fl  -II)  partition  I 
hence,  if  YeNfll),  Y  /  0  then  Ye  Rfl  -II).  Since  it  has  been  shown  that  every 
vector  in  Sfll)  is  also  in  Rfl  -II),  and  vice  versa,  the  lemma  is  proved. 


Theorem  6 

The  general  solution  of  the  system  of  homogeneous  linear  equations  AX'  =  0, 
where  A  is  m  ■  n ,  is 

\  (i  -  \"m. 


where  Y  is  an  arbitrary  n  .  l  vector  and  I  is  the  n  ii  identity  matrix. 

Proof:  Let  II  -  A«A. 

Since  A  AA"A  -  All,  rank  f A)  c  min  Lrank  fA),  rank  fll)]  (Ref  34,  p  75). 
Therefore  rank  fA)  -i  rank  fll). 

Similarly,  rank  fll)  s  rank  fA)  as  II  -  A  "A 
Therefore  rank  fA)  =  rank  'll). 

Also  II  -  =  (AMA)  f A M A )  =  A"f.AA"A)  =  A"A  =  II;  hence  II  is  idempotent  and 
consequently  Lemma  2  implies  that  nullity  fll)  =  rank  fl  -II)  Moreover  A  is 
in  n;  hence,  by  Lemma  1,  it  =  rank  fA)  +  nullity  fA),  so  that 

nullilv  (  \  I  II  r.ink  ( A ) 

II  r.ink  III ) 
nullilv  (II) 


Therefore 


nullilv  (  A)  -  rmik  fl  -  II) .  (1) 

But  A 'I  -II)  r-  \  -AA“A  -  0.  Hence  all  the  columns  of  fl  -II)  are  in  \  f  A ) .  More¬ 
over  Eq  1  implies  that  fl  -  II)  has  the  same  number  of  linearly  independent  col¬ 
umns  as  any  basis  for  \ f A ) .  Hence  the  columns  of  fl  —  II)  span  \ f A).  Therefore 
for  any  vector  U  e  \  f A)  there  exists  some  vector  Y  such  that  U  fl  -  II )  Y . 

Conversely  any  vector  of  the  form  fl  - II,  Y  is  in  NfA),  as  Afl  -  II)  Y  = 
fA  -  AAH.A)  Y  =  0.  Thus  it  has  been  shown  that  any  vector  A  is  in  \ f A )  (i.e.,  sat¬ 
isfies  AX  =  0)  if  and  only  if  there  exists  some  Y  such  that  \  =  f I  -AWA)Y  ,  and 
so  the  theorem  is  proved. 

Moreover  as  X  =  AHb  is  by  Definition  1  a  particular  solution  of  AX  =  b, 
whenever  AX  b  is  consistent  the  following  applies: 

Theorem  7 

The  general  solution  of  AX  -  b,  when  consistent,  is 

\  -  \"i.  i  (i  -  \"\n, 

where  Y  is  an  arbitrary  tul  matrix. 
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In  most  of  the  preceding  results  it  hrs  been  assumed  that  the  system  of 
linear  equations  AX  -  b  is  consistent.  The  following  theorem  tells  us  when  this 
assumption  is  valid. 

Theorem  8 

The  system  of  linear  equations  AX  =  b  is  consistent  if  and  only  if  AA#b  «=  b. 
Proof:  If  AA"b  -  b,  then  X  =-  A"b  is  a  solution  of  AX  =  b  implying  consistency. 
Conversely,  if  AX  b  is  consistent  then  AAttb  AAWAX  -  AX  =  b. 

Some  special  cases  of  A#  are  now  considered,  beginning  with  the  following: 
Definition  It— A11  will  be  called  a  “right  inverse”  of  A  if  AA#  =  I ,  the 
identity  matrix. 

Definition  17— A'1  will  be  called  a  “left  inverse”  of  A  if  A#A  =  I,  the  hxh 
identity  matrix. 

It  is  clear  from  Definition  14  that  any  right  or  left  inverse  is  also  a  g.i. 

In  Theorem  10  it  is  proved  that  the  converse  is  also  true;  i.e.,  any  g.i.  is  also 
a  right  or  left  inverse  if  such  an  inverse  exists. 

Theorem  9 

Let  A  be  an  in  n  matrix  with  m  ^  n. 

Then  A  has  a  right  inverse  if  and  only  if  rank  (A)  =  m. 

In  other  words,  A  must  have  full  row  rank  in  order  to  have  a  right  inverse. 
Proof:  Suppose  there  exists  an  A"  such  that  AA"  =  I.  Then  rank  (AA#)  = 
rank  (I)  =  in.  But  rank  f A  \")  =  »i  min  Lrank  f A) ,  rankfA#)]. 

Therefore  m  s  rank  f A). 

But  ni  a  rank  (A)  as  A  is  in  n;  so  rank  f A)  =  in. 

Conversely,  if  rank  f A)  =  in,  then  (AA1)-  1  exists  (Ref  35,  p  29)  and  so 
V*  =  (AA1)-  1  is  a  right  inverse  of  A. 

Corollary.  A  necessary  and  sufficient  condition  for  a  matrix  to  have  a 
left  inverse  is  that  it  have  full  column  rank. 

The  remark  made  following  Definition  17  will  now  be  proved. 

Theorem  10 

Let  \  be  in  •  n  witi.  rank  in. 

Then  AA"  =  I  for  any  g.i.  A"  of  A. 

Proof:  Let  A"  be  a  g.i.  of  A. 

Then  X  -  A"b  is  a  solution  of  AX  b  whenever  the  system  is  consistent. 

But  rank  (A)  =  m  implies  that  AX  b  is  consistent  for  any  bcE'".t 
Hence  AA"b~b  for  all  be  Fm. 

Consequently  AA"  -  I,  the  identity  matrix,  by  definition  of  I. 

Therefore  A"  is  a  right  inverse  of  A. 

Corollary.  If  A  has  full  column  rank,  then  A"  is  a  g.i.  of  A  if  and  only 
if  A"  is  a  left  inverse  of  A. 

Theorem  11 

Let  A  be  an  m  n  matrix  of  rank  in. 

tRank  (\)  -  m  means  that  A  has  m  linearly  independent  columns  and  so  some  subset 
of  the  columns  of  \  forms  a  basis  for  [m.  Hence  It  (A)  is  equal  to  [ffl. 
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Then  a  necessary  and  sufficient  condition  for  a  g.i.  V:  of  A  to  be  unique 
is  that  »n  -  n 

Proof:  Let  the  columns  of  A"  be  a  ,  j  =  1,  .  .  .  ,  ti. 

Then  A  V!  I  implies  that  An,  =  r, ,  wnere  c,  is  the  tn  /  I  vector  with  unity 
in  the  |th  row  and  zeros  elsewhere. 

But  the  general  solution  of  the  system  Aa(  =  c,  is,  by  Theorem  7,  e,  = 

A*1  c(  +  (  where  f  is  an  arbitrary  vector  in  \f  A),  and  V'  is  any  g.i.  of  \. 

Hence  will  be  unique  only  if  f  is  unique. 

But  if  f  is  unique  and  f  c  \  (A) ,  nullity  'A)  =  0. 

Since  it  =  rank  (A)  +  nullity  (A)  by  Lemma  1,  ran!:  ')  =  »i. 

But,  by  assumption,  rank  f  A)  =  m. 

Therefore  a(,  and  hence  A",  being  unique,  implies  m  =n. 

Conversely,  if  m  =  ri,  then  rank  fA)  =  ti  =  m.  By  the  previous  theorem  and 
its  corollary,  this  means  that  A  has  both  a  left  and  a  right  inverse;  hence  V  \~ 
is  the  unique  right  inverse  of  A. 

[To  see  this,  let  A"  and  A"  be  left  and  right  inverses,  respectively,  of  A 

Then 

\'l  \  v;  -  ( tj  u  \::  - 1  v;  -  \:; 

and 

v,  \  \|  *  \  \s»  -  'T 1  'i  • 

Therefore 


and  the  right  and  left  inverses  are  equal.  Since  this  must  hold  for  all  right  and 
left  inverses  of  A,  these  inverses  must  be  unique.  The  regular  inverse  A-1  of 
A  is  defined  as  this  unique  right  and  left  inverse.  „ 

Corollary.  If  the  right  inverse  of  A  is  unique,  then  A"  =  A-1 . 

Next,  the  theorem  showing  how  to  generate  all  right  inverses  of  a  matrix 
from  any  given  one  is  proved. 

Theorem  12 

Let  A  be  an  m  •  ii  matrix  with  rank  rn. 

Let  AA"  I. 

Then  AA!i  =  I  if  and  only  if  there  exists  an  n  ■  m  matrix  f  such  that  \“  A  I 
where  each  column  f(,  i  =  1,  .  .  .  ,  m,  of  |  is  in  \  f  \) 

Proof:  Let  f  =  (fi,  .  .  .  ,  frM)  be  an  ti  -m  matrix  such  that  f,e  VA)  ,1  =  1, 

.  .  .  ,  m. 

Let  A"  =  A"  .  f 
Then 

t't"  -  \  ( \  •  |  >  -  J  •  \  I 
'  •  ‘  M  i . M  J 


as  A  f  f  =  0  for  all  i  because  f ,  c  N  f  A). 

Conversely,  if  AA,!  =  I,  then  we  can  define  I  A"  -  A"  and  \|  =  AA"  -  VA  • 
1-1=0.  So  each  column  of  f  is  in  VA). 
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Corollary.  Let  A#A  -- 1. 

Then  AHA  =  I  if  and  only  if  A"  =  AH  +  f,  where  earn  row  of  I  it>  a  vector 
in  \(AT). 

Theorem  13 

Let  A  be  an  m  >  n  matrix  of  rank  n. 

Then  AX  =  b,  b  /  0,  has  either 

(a)  no  solution  if  b«,R'A),  or 

(b)  the  unique  solution  X  A"b  if  b  -  R'A)  and  A  "  is  any  left  inverse  of  A. 

Proof:  (a)  This  follows  directly  from  the  definition  of  R(A). 

(b)  Because  A  has  rank  n,  the  system  AX  =  b  is  a  system  of  m  equations  ex¬ 
actly  n  of  which  are  linearly  independent.  Thus  AX  =  b  has  a  set  of  n  independent 
equations  in  n  unknowns  and  so  has  a  unique  solution.  Since  X  -  A#b  is  a  solu¬ 
tion  of  AX  =  b,  the  theorem  is  proved. 

The  fact  that  AHb  is  unique  regardless  of  what  An  is  chosen  can  also  be 
seen,  as  follows:  It  is  known  that  the  general  solution  of  AX  b  is  X  =  AHb  * 
fl  -  A ** A )  Y  ,  where  Y  is  arbitrary.  But  rank  f A)  =  n  so  that  A"  is  a  left  inverse, 
by  the  corollary  to  Theorem  10;  hence  I  -  A  "A  =  0,  and  the  general  solution  of 
AX  =  b  is  X  -  A«b. 

Let  A#  be  any  other  left  inverse  of  A. 

Then,  by  the  corollary  to  Theorem  12,  AM  -  A"  f  f,  where  each  row  f,  of 
f  is  in  X  (A  1 ).  Therefore 


But  f,t\'AT)  and  b  R  ^  \ )  ;  hence  f,b  =  0,  i  =  1,  .  .  .  ,  n.t 

Therefore  Af,b  AMb  X  is  unique  as  A'!  was  any  other  left  inverse  of  A. 

Theorem  14 

Let  A  be  an  m  u  matrix. 

Let  rank  \  n  .  Then  (a)  the  equation  to  (  \  „ CT  has  the  solution  cot  crAH, 
where  A"  is  any  left  inverse  of  A ;  and  (b)  to  r  is  unique  if  and  only  if  n  m. 

Proof:  (a)  If  to  T  =  c  rA#,  then  co  rA  =  c 1 A  *  A  =  c  l  .  Hence  to  r  =  cT  \ #  is  a 

solution  of  to  T  A  =  CT. 

(b)  The  general  solution  of  to 1 A  =  c1  is  to r  =  cT  A*  +  f ,  where  f  e  \(AT). 
Therefore  co  r  is  unique  if  and  only  if  f  is  unique. 

But  f  is  unique  if  and  only  if  nullity  'A  r)  =  0,  and  nullity  'A  *)  =  0  if  and 
only  if  rank  '  \)  m. 

Therefore  to  r  is  unique  if  and  only  if  n  =m. 

+This  follows  from  the  fact  that  \\  -  h  and  f ,  r  \  ( A j  — ►  f ,  \  -  0;  hence 

f,  h  =  f,  A  \  0  \  -  0. 
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4.  COMPUTATION  OF 


In  order  to  avoid  unnecessary  complications  in  the  algebraic  treatment 
in  this  section,  4  will  be  made  a  square  matrix  by  adding  rows  or  columns  of 
zeros,  whichever  is  necessary.  The  necessary  change  in  the  system  of  linear 
equation  A.X  -b  is  to  extend  b  by  a  number  of  zeros  if  rows  of  zeros  are  added 
to  A,  and  to  extend  X  by  a  number  of  variables  if  columns  of  zeros  are  added 
to  A.  These  changes  do  not  in  any  way  alter  the  given  system  of  equations  since 
adding  rows  results  in  adding  the  identity  0  =  0  to  the  given  system,  whereas 
adding  columns  always  makes  the  new  variables  in  X  appear  in  the  system  with 
the  coefficient  0. 

Moreover  the  g.i.  for  the  original  matr  ix  can  always  be  obtained  from  the 
g.i.  of  the  extended  square  matrix  by  dropping  some  rows  or  columns.  For  if 
A  ,  the  original  matrix,  has  m  •s  n  ,  and 

'  (o\) 


is  the  extended  matrix,  where  0A  is  the  (n  -  m)  x  rt  matrix  of  zeros,  then 
AAW4  -  \  means  that 


cxor-x;) 


where  \H  =  (Q,,  Q2) , Q j  is  «>m,  Q2  is  n  >'n  -m),  and  0i  and  02  are  matrices 
all  the  elements  of  which  are  0. 

But 


Vu 


means  that  A  =  AQjA.  Hence  Q|  is  a  g.i.  of  A  and  is  obtained  from  by 
dropping  Q  >,  the  last  n-m  columns  of  A". 

Similarly,  if  A  has  m  >n  ,  m  -n  columns  of  zeros  are  added  to  A  to  get  the 
extended  matrix  t  and  A"  is  obtained  by  dropping  the  last  m-n  rows  from  A". 

Thus  the  system  AX  -  b  where  A  is  square  will  be  considered  without  loss 
of  generality. 

Definition  18— The  following  matrix  operations  are  called  “elementary 
row  operations”: 

(a)  interchange  of  two  rows; 

(b)  multiplication  of  a  row  by  a  non-zero  scalar;  and 

(c)  replacement  of  the  i  th  row  by  the  sum  of  the  i  th  row  and  k  times 
the  |  th  row,  where  i  /  j  and  fe  is  any  scalar. 

Definition  19— A  matrix  H  is  said  to  be  “row-equivalent”  to  a  matrix  A 
if  II  can  be  obtained  by  performing  a  succession  of  elementary  row  operations 
on  A. 

From  Perlis  (Ref  33,  p  42)  we  have  the  result  that  for  any  square  matrix 
A  there  exists  a  matrix  II  that  is  row-equivalent  to  \  and  such  that: 

(a)  all  diagonal  elements  of  II  are  zeros  or  ones; 

(b)  when  the  i  th  diagonal  element  is  0,  all  elements  in  the  i  th  row 
are  0  and  all  elements  in  the  i th  column  below  the  diagonal  element  are  0;  and 


13 


4 


(c)  when  the  jth  diagonal  element  is  1,  the  ith  column  is  a  unit  vector 
and  all  elements  preceding  the  ith  in  the  ith  row  are  0. 

Moreover,  since  each  elementary  row  operation  is  the  result  of  premulti¬ 
plying  A  by  a  nonsingular  matrix,  it  follows  from  Perlis  that  there  exists  an 
n  x  n  nonsingular  matrix  Q  such  that  H  =  QA  (see  also  Ref  36,  p  24).  This  leads 
to  the  following  theorem: 

Theorem  15 

Let  A,  H,  Q  be  as  defined  above. 

Then  (i)  H  is  idempotent. 

(ii)  AH  -  A. 

(iii)  4QA  A. 

Hence  0  is  a  Rao  g.i.  of  A. 

Proof:  (i)  This  follows  directly  from  properties  a,  b,  and  c  following 
Definition  19.  For  if  h; ,  the  ith  row  of  H,  has  0  as  its  diagonal  element,  then 

the  matrix  D  =  HI!  =  fdl()  has  h,hi  =  di(  =  0,  j  =  1,  .  .  .  ,  n,  since  h,  is  a  zero 

vector  where  hi  is  the  ith  column  of  H.  If  hi  has  1  as  its  diagonal  element  it 

is  a  unit  vector;  hence  h,h i  =  dl(  =  h,, ,  i  =  1,  .  .  .  ,  n,  the  ijth  element  of  H. 
Since,  by  a,  every  diagonal  element  of  H  is  0  or  1,  D  =  H  and  so  i  is  proved. 

(ii)  Let  H  =  QA  where  Q  is  nonsingular. 

Hence 

A  -Q_1  II 

so  that 

All  -Q“'  II2  =  Q“'  II  In  i 

-  \. 

(iii)  Since  QA  =  H,  AQA  =  AH  =  A  by  ii. 

Thus  by  Definition  14,  Q  is  a  Rao  g.i.  of  A. 

Since  Q  is  the  product  of  all  the  matrices  that  give  the  elementary  row 
operations  involved  in  the  reduction  of  A  to  H,  Q  can  be  found  by  performing, 
in  turn,  exactly  the  same  elementary  row  operations  on  the  mi  identity  matrix 
I  that  we  perform  on  A  in  reducing  it  to  H.  In  other  words,  beginning  with  A 
and  I,  and  applying  the  method  of  sweep-out  and  interchange  of  rows,  if  nec¬ 
essary,  to  bring  unities  to  the  diagonal,  to  reduce  A  to  H,  (i.e. ,  using  the  Gaus¬ 
sian  elimination  technique)  Q  will  be  the  matrix  obtained  by  performing  in  order 
exactly  the  same  operations  on  I  that  are  performed  on  A.  Thus  a  g.i.  con¬ 
sidered  above  is  computed  in  the  same  way  as  a  regular  inverse  when  it  exists. 


5.  EXAMPLE  OF  a"  AND  A* 

The  technique  for  computing  Aw,  discussed  in  the  preceding  section,  is 
illustrated  here.  The  computation  may  be  abridged  to  a  large  extent  by  omitting 
some  intermediate  steps,  but  it  is  presented  here  in  full  to  illustrate  the  method. 
Let 
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So  that 


In  these  computations  the  second  block  was  obtained  by  taking  the  2  (circled 
in  the  first  block)  as  a  pivot  and  then,  by  successive  subtraction,  sweeping  out 
the  first  column.  The  third  block  was  obtained  by  pivoting  on  the  l/2  (circled  in 
the  second  block)  and  sweeping  out  the  second  column.  The  last  block  is  obtained 
simply  by  interchanging  third-block  rows  2  and  3. 

It  is  easily  verified  that  A"  given  above  is  a  g.i.  inverse  of  A.  Aw  is  not, 
however,  unique.  For  example,  another  g.i.  of  A  is 

\*  /  o  i  <•  \ 


The  computation  of  A+  is  presented  below  in  order  to  show  the  increased 
computation  required  in  such  an  operation.  This  method  of  finding  A*  is  con¬ 
tained  in  Ref  13. 

The  formula  for  A+  is 


A* 


<)•> 


i 


02  X2  is  the  2x2  null  matrix.  A  and  E  are  defined  by  Ben-Israel  and  Wersan.13 
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Sweep-out  techniques  are  then  used  on  ATA  and  AT  to  get 
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The  second  block  was  obtained  by  pivoting  on  the  6  (circled)  and  then  sweeping 
out  the  first  column.  The  last  block  was  obtained  by  pivoting  on  the  l/2  (circled) 
and  sweeping  out  the  second  column. 

Then,  by  the  definitions  of  A  and  EAT  in  the  Ben-Israel  and  Wersan  paper, 


It  can  be  verified  that  A+  does  indeed  satisfy  Definition  3.  Thus  the  great 
increase  in  computation  that  results  when  A4  is  computed  instead  of  Aw  can  be 


13 
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seen.  Since  the  linear-programming  results  developed  below  hold  for  V  as 
well  as  4f ,  it  will  be  more  efficient  to  use  4“  in  solving  linear  programming 
problems. 


6.  THE  SIMPLEX  ALGORITHM 


Consider  the  following  linear  programming  problem: 
maximize 


i 


i 


A 


subject  to 

l-A-P,, 

A  •  0 

where  P  is  an  m  vn  matrix  and  the  columns  of  P  are  P. ,  f  =  1,  .  .  .  ,  n . 

Let  L  be  the  subspace  of  E"1  that  is  spanned  by  the  vectors  (P  j,  .  .  .  ,  P„) 

Thus  L  is  the  set  of  all  linear  combinations  of  P . ,  j  =  1,  .  .  .  ,  n . 

Let  B  be  a  basis  for  L.  Then  each  vector  in  B  is  in  L;  and  B  has,  at 

most,  m  elements,  as  L  is  subspace  of  E>".  It  will  be  assumed  that  B  has  s 
elements.  Thus  B  is  an  m  x  s  matrix  with  s  *>  m,  and  because  of  the  linear  in¬ 
dependence  of  the  vectors  in  a  basis,  rank  fB)  -  s. 

Let  B  -=  [P,  :  i  el }  and  renumber  P, ,  j  ■  1,  .  .  .  ,  n  so  that  B  =  (P, ,  .  .  .  ,  Ps). 
Then,  by  definition  of  a  basis,  any  P(,  j  =  1,  .  .  .  ,n  can  be  expressed 
uniquely  as  a  linear  combination  of  the  basis  vectors  as  follows: 


where 


Suppose  the  basis  is  changed  by  removing  Pr  from  B  and  inserting  Pb  in 
its  place.  Then,  provided  xrk  t  0,  the  set  of  vectors  {P,  :  i  d  ,  i  i  r  ]  and  Pb 
will  form  another  basis  for  L . 

This  follows  from  the  easily  proved  Lemma  3. 

Lemma  3.  Let  L  be  the  vector  space  spanned  by  [u],  u2,  .  .  .  ,  u„). 

Let  H  ,  cL  be  given  by 

M 

H  |  =  A|  Uj  4  i.  u f  with  A|  4  0. 


Then  (a)  the  space  spanned  by  {IV  j ,  u  o ,  *  •  .  ,  un]  is  L  and 

(b)  if  {« i ,  u 2 »  •  •  •  ,  are  linearly  independent  ( i.e. ,  form  a  basis 
for  L)  then  so  are  [IV],  u2»  •  •  •  >  unJ  (i»e->  the  latter  set  is  also  a  basis  for  L  ). 

1  j  new  basis  will  be  denoted  by  B.  That  is,  B  —  fP  i . Pr  _  i ,  Pj. .  Pr .  |. 

.  .  .  ,  ) 

Dei  ne 

7r  . 1  *rk’ . “  ( *  s  It 
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From  Eq  2  we  have 


Hence 


or,  in  matrix  notation 


and 


If  I 
I  -ft 


|'r  V  |»,  /  *ifc  \  , 

U|  y  * rk’  J  *rk 

i  /  r 


I’r  I!,, 

I’,  -  Hi’,,  1 1 1,  i  t  r , 


(3) 

(4) 


where  c,  is  the  s  x  1  vector  with  +  1  in  the  ith  row  and  zeros  elsewhere. 

Let  E  be  the  s  matrix  defined  by  E  =  (ct  t  .  .  .  ,  cr_  | ,  rj,cr  4  |  ,  .  .  .  ,  cs). 
Then  E  is  nonsingular,  from  the  definitions  of  r?  and  e,.t 
Also  from  Eqs  3  and  4 

It  (P  | . I\ . IM  (Me , . Hi/ . Mcs )  fir:. 

But  B  has  full  column  rank  and  hence,  by  the  corollary  to  Theorem  10, 1 
is  a  left  inverse. 

Therefore 

lt"»  l  M”fic 


or 

IE-1  E*1  MrtMEE  “ 1 


Hence 

EE-1  -I  CM  "fi 

so  that  a  left  inverse  of  the  new  basis  fi  is  given  by  EB",  i.e., 

fi"  EH".  (5) 

Definition  20— Given  the  system  AX  =  b,  where  A  is  m--n  and  rank  f.A)  *•  s  , 
and  any  m  >  s  basis  B  whose  columns  are  columns  of  A,  then  the  solution  X  „  of 
the  system  B\(J  =  b  is  called  the  “basic  feasible  solution”  for  the  system  AX  -  b 
with  basis  B. 

Now  suppose  that  the  basic  feasible  solution  corresponding  to  the  basis 
B  isX().  Then  BX()  =  P() ;  hence 

(6) 

and,  by  Theorem  13,  X„  is  unique. 

tThis  is  true  because  the  rth  diagonal  element  of  E  is  l/xrk  i  0,  and  all  columns 
of  E  other  than  the  rth  are  unit  vectors. 
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Also  because  B  is  a  basis,  it  is  known  from  Eq  2  that  li\( 
.  .  .  ,  it ,  and  hence 


\  '<"'V 


r,,  I  =  1, 


(7) 


Furthermore  the  \(  are  unique. 

Then  for  the  new  basis  fi  it  follows  that  1(,  =  B"P„  and  =  fi  ' P (  ,  j  =  1, 
.  .  .  ,n  are  the  unique  values  of  A|j  and  X, .  Hence,  using  Eq  5. 

am  I  I  A„  (8) 

and 

N,  M»"l,|  IN,,  i  I..  ..ii.  (9) 


Equations  8  and  9  represent  the  simplex  iteration  formula  of  linear  programming 
theory  because,  if  Eq  9  is  put  in  component  form, 


\ 


I 


f\  - 
■ .  i 


or 


Therefore 


i . ii 


which  is  the  ordinary  equation  used  in  the  simplex  algorithm  (Ref  37,  Vol  I, 
p  142). 

Similarly 

Al  Al  *  *11: 

Thus  it  has  been  shown  that  given  any  basis  B  and  the  corresponding 
simplex  tableau,  whose  columns  are  given  by  Eqs  6  and  7,  all  succeeding 
tableaus  computed  using  the  simplex  algorithm  will  be  the  same  as  those  ob¬ 
tained  by  computing  B"  at  each  iteration.  Hence  the  simplex  algorithm  works 
perfectly  well  when  the  basis  is  not  square. 

From  simplex  theory  it  is  also  known  that  for  an  optimal  basis  B  the  m,  1 
vector  of  dual  variables  a;  is  the  solution  of  the  system  of  equations 

..'ll  «  /,.  ul..-r.-  i’,1,  h,  I’,.  HI.  (10) 


This  is  true  because,  from  Eq  10,  oc  1  BX„  =  cjf  A f{  =  optimal  value  of  objective 
function  =  ;,  where  X()  is  the  optimal  basic  feasible  solution.  From  the  fact 
that  BXu  =  P(),  u^P,,  =  ;  which,  by  the  dual  theorem  (Ref  37,  Vol  I,  Chap. VI), 
means  that  u:  is  optimal  for  the  dual  of  our  given  problem. 

But  Eq  10  says  that  ce  1  =  c (r,  B!i,  which,  by  Theorem  14,  is  unique  if  and 
only  if  B  is  square.  Thus  it  has  been  shown  that  in  a  distribution  model  the 
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optimal  solution  of  the  dual  is  not  unique,  since  the  model  has  one  redundant 
constraint;  hence  B  is  not  square. 


7.  THE  MODIFIED  SIMPLEX  METHOD 

The  linear  programming  problem  given  at  the  start  of  Section  6  will  be 
reconsidered  here.  The  fundamental  equation  of  the  modified  simplex  method 
(Ref  37,  Vol  II,  pp  471-74)  is 


1)  0 

\  ■ 

m  ■  1 

Ifl  ■  S  HI  ■  1 

S-  1 

. 

-c  1  1 

. 

i 

1  li  1 

1 

I-  1 

1  ■  S  I-  1 

1  ■  1 

(ID 


where  Pj  is  any  column  of  P ,  B  is  a  basis  for  the  space  spanned  by  the  columns 
of  P,  X.  is  the  expression  of  P,  in  terms  of  the  basis,  c()  is  the  vector  of  costs 
of  the  P,  di,  and  z(  is  defined  by  z,  =  c/f  xy 

Let  B"  be  any  left  inverse  of  B.  Then  if  the  mxl  vector  w  is  defined  by 
ccTB  sfT  (i.e.,  cc  I  =  c  '  B") 


!)"  (1 

II  0 

1 

0 

1 

-<'n  ' 

l 

i  o 


SvS 

s  • 

0 

1 

Is 

1 . 

Using  Eq  11  the  following  is  obtained: 


■rc, 


li"  » 


(12) 


where  u.' T  =  cT  B". 

The  fundamental  Eq  11  can  be  generalized  to  the  whole  tableau  by  defining 
X0  =  X  and  z()  -  c()  =  z  =  value  of  objective  function  and  Co  =  0. 

Thus  the  entire  (s  +  1)  x  (n  +  1)  simplex  tableau,  where  the  Zj  -  c j  appear 
in  the  s  +  1st  row,  can  be  generated  by  Eq  13: 

\>  •  ^  i . \i 

' ’  '  1  I . :n"1  n 


li"  () 

s  ■ in  s  •  1 

c'u"  I 
l-l  l-l 


>’.)•  . 

. 


(13) 


Consequently  given  any  m  /s  matrix  B,  which  is  a  basis  for  the  space 
spanned  by  the  columns  of  P,  B“  can  be  computed  and  hence  the  entire  simplex 
tableau,  including  the  z,  -  c(  row,  by  using  Eq  13.  Since  it  has  already  been 
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shown  that  the  calculation  of  B»  from  B»  can  be  done  by  using  the  usual  simplex 
formulas,  it  can  be  concluded  that  the  modified  simplex  method  will  work  for 
any  basis,  regardless  of  whether  the  basis  is  square. 


8.  EXAMPLE 

To  illustrate  the  theory  the  following  network  problem  will  be  solved.  An 
arbitrary  basis  B  will  be  selected  and  computed  and  Eq  12  will  be  used  to  gen¬ 
erate  the  initial  simplex  tableau.  The  simplex  algorithm  is  then  used  until  an 
optimal  solution  is  reached.  Finally  a  left  inverse  of  the  optimal  basis  is  com¬ 
puted  and  used  to  find  the  optimal  vector  of  dual  variables  and  hence  the  optimal 
value  of  the  dual  objective  function.  Diagrammatically  the  network  is  as  shown 
in  Fig.  1. 


Fig.  1 — Diagram  of  the  Network  Problem 

Arrow  shows  direction  of  link;  number  beside 
arrow  is  the  c of  the  link.  Dotted  arrow  in¬ 
dicates  link  used  to  form  initiol  basis. 


The  problem  to  be  solved  is: 
maximize 


lih„i  *  20x 


02 


30  x 


It 


20x 


1 1 


10 1 ,,  t  20  x. 


20jf:!()  *  M)  i 


15 


20  x 


56 


subject  to 


— *02 

*01  ”  *13  “  *it 

*0°  —*0.1  —  *24 


-  I 

-  0 
0 


1  *23 


It 


t  *2| 


*36 


0 

0 

0 

1 


(14) 


0.  all  l,  J, 


where  x  l}  is  the  amount  programmed  to  flow  from  node  j  to  node  j . 
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In  the  incidence  matrix  for  the  problem,  “+”  and  are  used  to  denote 
“+1”  and  “-1"  and  all  blanks  represent  0. 


The  initial  basis  matrix  is  B  =  (P,  ,  p., ,  P:!,  p- ,  p7 ,  PH).  is  computed  as 
follows: 

Column  Column 
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Column 


Column 


23 


These  simplex  iterations  then  lead  to  the  following  optimal  tableau: 
First  iteration:  Insert  P9  into  the  basis  and  remove  P7 . 

Second  iteration:  Insert  Pt  into  the  basis  and  remove  P3. 

Third  iteration:  Insert  P6  into  the  basis  and  remove  Pf). 

Optimal  tableau: 


Thus  the  optimal  solution  is:  x02  =  +  1,  x21  =  +  1,  xJ5  =  +  1,  xfl6  =  +  1; 
and  z  =  100  is  the  optimal  vnlue  of  the  objective  function.  Since  the  optimal 
basis  B*  =  (P , ,  P 2,  P  t,  Pf,,  P9  ,  P8  )  we  find  that 


I  2  i  \  :>  6 
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so  that  u>  *T,  the  optimal  dual  solution,  is  given  by 

r  C,1,*  B*"  (10,  20,  ,10,  20.  20,  tO)B‘" 

(-100,  -00,  -80.  -,i0,  -60.  -20,  0). 

Hence  w*!  p()  =  100  =  optimal  value  of  dual  objective  function.  Thus  the  values 
of  the  objective  functions  of  the  primal  and  dual  problems  are  equal  at  the 
optimum. 
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